Datacenters have experienced rapid growths, and the growth rate is expected to accelerate. The torrid development is fueled by an increasing demand, and made possible by reduced costs of the components of datacenters. Datacenter are chiefly constructed from processing nodes, storage nodes, and networks that connects the processing nodes and the storage nodes. Both the processing nodes and the storage nodes have become smaller, less expensive, and more energy efficient, allowing datacenters to pack more processing and storage nodes into smaller spaces to meet the increasing demand for data processing and storage. As the processing nodes consume more and more data at higher rates, and retrieve and store the data from and into storage nodes, the networks must transmit more and more data at higher speeds between increasing numbers of connections. As a result, the costs of the networks have become significant, in relationship to the falling costs of the processing and storage nodes. One estimate puts the cost of the networks at ˜50% of new datacenters.
In a traditional datacenter, processing nodes are typically connected via a single primary network. Secondary networks, if any, are primarily used for administrative purpose and is not a topic of discussion here. Each processing node may have one or more locally attached long term storage devices such as hard disks or solid state disks. A processing node accesses its long term storage device to satisfy its internal needs and often on behalf of a system-wide distributed storage system. A number of processing nodes, each with one or more long term storage devices are packaged in a processing module. The computing power of the datacenter is scaled up primarily by adding processing modules. This construction framework places a heavy demand on the primary high-speed network, since every processing node added relies on the primary high-speed network to communicate with existing processing nodes. The capacity of the high-speed network has to increase in proportion to the numbers of processor nodes added. Because processing nodes are becoming faster and less expensive, while fast connectivity is getting more expensive, high-speed network costs have become the bottleneck that impedes the scaling up of datacenter computing capacity at low cost.
Facebook has set out the Open Compute Project, aiming to develop datacenter servers that are both energy- and cost-efficient. The solutions that the Open Compute Project promotes includes vanity-free design of hardware, open vault storage building blocks, mechanical mounting system, and high disk densities. The result of these efforts are datacenters made of vanity-free servers that are up to 38% more energy-efficient and up to 24% less expensive to build and run than traditional server hardware. However, the solutions practiced in the Open Compute Project amounts to optimization of packing processor nodes. The fundamental dichotomy between a processing function and a storage function, along with the resulting network traffic between the processing nodes that dedicated to computing and storage devices dedicated for storage, remains unchanged.
Nutanix has developed Nutanix Virtual Computing Platform that incorporates a high speed storage (Server Flash) and low speed storage (Hard Disk Storage) locally to processing nodes, to increase the speed and efficiency of computing for datacenters. However, no fundamental network improvement is revealed.
Thus there remains a need for keeping the demand for fast connectivity in datacenter at bay while adding more processing nodes to accommodate the ever-increasing need for more computing power. Preferably, when new processing nodes are added into an existing computer system, the new processing nodes contain their networking functionality and do not require dedicated network equipment to be installed, so that computing power and network capacity grow along with the addition of the processing nodes.